Papers by James Seale Smith
FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing (2025.naacl-long)
Copied to clipboard
James Seale Smith, Chi-Heng Lin, Shikhar Tuli, Haris Jeelani, Shangqian Gao, Yilin Shen, Hongxia Jin, Yen-Chang Hsu
| Challenge: | Empirical evaluations demonstrate substantial performance gains over existing methods . |
| Approach: | They propose a method to prune LLMs that selectively prunes model blocks based on an importance score and replaces them with a low-parameter replacement strategy. |
| Outcome: | The proposed method achieves state-of-the-art performance on 5/6 and 6/6 benchmarks with a compression rate of 30% and 40%. |